The vast majority of users of computing devices interact with those computing devices through user input peripherals that are controlled by a user's hands; including, for example, the ubiquitous keyboard and mouse user input peripherals, as well as, more recently, touch-based peripherals such as trackpads, touch sensitive screens, and devices that can accept multi-touch input. Similarly, the vast majority of users of computing devices receive feedback from those computing devices through graphical user interfaces that are seen by the users. Certain individuals, who still desire to utilize and interact with computing devices, may not have the capability to utilize common input peripherals or to see commonly utilized graphical user interfaces. For example, those individuals who are blind cannot perceive a visually presented graphical user interface. Similarly, those individuals that have a physical handicap, such as damaged or missing arms and hands, cannot accurately utilize common input peripherals.
Traditionally, those users whose physical limitations prevent them from interacting with computing devices in a standard way have been able to interact with computing devices through specialized hardware and software designed to address those users' needs. For example, users who are blind, or who cannot otherwise accurately perceive a visually presented graphical user interface, can utilize a screen reader that can present content from a computing device, which would typically be displayed in a visual manner, instead in an auditory manner. Similarly, users having a physical handicap that prevents them from accurately utilizing common input peripherals, can instead utilize non-traditional input peripherals such as an eye tracker that can enable a user to control certain aspects of the computing device through movement of their eyes, or a so-called “sip-n-puff” device that can enable a user to control certain aspects of the computing device through the movement of their mouth or tongue.
The technology that enables users whose physical limitations prevent them from interacting with computing devices in a standard way to still be able to interact with those computing devices is commonly known as “assistive technology”, and includes the above-mentioned screen readers, eye trackers, and other such devices. Unfortunately, as computing interfaces have become more complex, assistive technology has struggled to keep up. For example, due to more complex, and more capable, input peripherals, such as peripherals that can accept multi-touch input, modern computing user interfaces can, simply, comprise too much information to be meaningfully consumed through assistive technology such as a screen reader.